1. Field of the Invention
The present invention relates to generation of pattern image data projected onto a target in a stereo method.
2. Description of the Related Art
Conventionally there is well known “stereo method.” In the stereo method, three-dimensional shape information on the measurement target or distance information to the measurement target is determined using two images which are obtained by imaging the measurement target from two different positions. In the stereo method, a depth distance is computed based on a triangulation principle by utilizing imaging position shift (parallax) of the measurement target which is generated between two taken images.
In the stereo method, in order to determine the parallax, it is necessary to search points (corresponding points) which correspond to each other in the taken images. The search is called “stereo correspondence problem.” Window-based matching can be cited as an example of the well-known stereo correspondence problem technique.
Process contents of the window-based matching will be described below with reference to FIG. 12.
First one of the taken images is set at a first image G1. In the first image G1, a reference pixel P1 is set to recognize a window W1 corresponding to the reference pixel P1. The window W1 is a pixel region including the reference pixel P1 and a set of pixels having predetermined positional relationships with respect to the reference pixel P1. In FIG. 12, the window W1 is the pixel region which is formed by three-by-three pixels centered around the reference pixel P1.
Then, the other taken image is set at second image G2, and one pixel located in the second image G2 is set at a target pixel P2 to recognize a window W2 corresponding to the target pixel P2. The window W2 is a pixel region including the reference pixel P2 and a set of pixels having predetermined positional relationships, determined in the same way as the window W1, with respect to the reference pixel P2.
A normalized correlation function C expressed by a formula (1) is computed using brightness values I1(i) of the pixels constituting the window W1 and brightness values I2(i) of the pixels constituting the window W2.
                    C        =                              (                                                            ∑                                      i                    =                    0                                                        W                    -                    1                                                  ⁢                                                                            I                      1                                        ⁡                                          (                      i                      )                                                        ⁢                                                            I                      2                                        ⁡                                          (                      i                      )                                                                                  -                                                1                  W                                ⁢                                                      ∑                                          i                      =                      0                                                              W                      -                      1                                                        ⁢                                                                                    I                        1                                            ⁡                                              (                        i                        )                                                              ⁢                                                                  ∑                                                  i                          =                          0                                                                          W                          -                          1                                                                    ⁢                                                                        I                          2                                                ⁡                                                  (                          i                          )                                                                                                                                          )                                                                                                    (                                                                                            ∑                                                      i                            =                            0                                                    W                                                ⁢                                                                                                            I                              1                                                        ⁡                                                          (                              i                              )                                                                                ⁢                                                                                    I                              1                                                        ⁡                                                          (                              i                              )                                                                                                                          -                                                                        1                          W                                                ⁢                                                                              ∑                                                          i                              =                              0                                                                                      W                              -                              1                                                                                ⁢                                                                                                                    I                                1                                                            ⁡                                                              (                                i                                )                                                                                      ⁢                                                                                          ∑                                                                  i                                  =                                  0                                                                                                  W                                  -                                  1                                                                                            ⁢                                                                                                I                                  1                                                                ⁡                                                                  (                                  i                                  )                                                                                                                                                                                                          )                                                        1                    2                                                                                                                                            (                                                                                            ∑                                                      i                            =                            0                                                                                W                            -                            1                                                                          ⁢                                                                                                            I                              2                                                        ⁡                                                          (                              i                              )                                                                                ⁢                                                                                    I                              2                                                        ⁡                                                          (                              i                              )                                                                                                                          -                                                                        1                          W                                                ⁢                                                                              ∑                                                          i                              =                              0                                                                                      W                              -                              1                                                                                ⁢                                                                                                                    I                                2                                                            ⁡                                                              (                                i                                )                                                                                      ⁢                                                                                          ∑                                                                  i                                  =                                  0                                                                                                  W                                  -                                  1                                                                                            ⁢                                                                                                I                                  2                                                                ⁡                                                                  (                                  i                                  )                                                                                                                                                                                                          )                                                        1                    2                                                                                                          [                  Formula          ⁢                                          ⁢          1                ]            
I1(i), I2(i): Brightness values in pixel position of cameras 1 and 2
W: Window size
In the formula (1), “i” is an identification number for identifying the pixels constituting the windows W1 and W2 by the positions of the pixels, and the identification number is set based on the same rule between the windows W1 and W2. That is, in the two pixels having the same identification number between the windows W1 and W2, the position in the window W1 coincides with the position in the window W2.
In the formula (1), “W” is the number of pixels (window size) constituting each of the windows W1 and W2. In FIG. 12, “W” is 3×3=9.
The normalized correlation function C computed by the formula (1) is a value which indicates similarity (ranging from −1 to 1) of the target pixel P2 to the reference pixel P1. A possibility of a correspondence correlation is increased between the pixels, as the value of the normalized correlation function C is increased (as the value is brought close to 1).
Therefore, the pixels of the second image G2 are sequentially set at the target pixel P2, the normalized correlation function C concerning each pixel is computed by the same process, and the pixel whose normalized correlation function C becomes the maximum is regarded as the pixel corresponding to the reference pixel P1 and set at the corresponding point.
The technique using the formula (1) is called normalized correlation method, and the normalized correlation method is a typical technique for the stereo correspondence problem.
In the normalized correlation method, even if the brightness values I1(i) (i=1 to w) are changed to I1′(i)=αI1(i)+β (α and β are a real number), the same result can be obtained. That is, the value of the normalized correlation function C computed from the brightness values I1(i),I2(i) (i=1 to w) becomes equal to the value of the normalized correlation function C′ computed from the brightness values I1′(i),I2(i).
Therefore, even if linear shift is generated between the brightness values in the images due to a difference in output characteristics between a camera 1 taking the first image G1 and a camera 2 taking the second image G2 or the like, the appropriate stereo correspondence problem can be performed by absorbing the linear shift.
The window-based matching is a technique of identifying the correspondence correlation based on distribution characteristics of the pixel brightness value. Therefore, for example, as in the case where the measurement target is an object having a monochrome surface, when the measurement target has an even surface brightness distribution to hardly generate the change in brightness value in the taken image, it is difficult to perform the stereo correspondence problem.
On the other hand, a technique for solving the above problem describes projecting a predetermined projecting pattern onto the surface of the measurement target to put the pattern on the surface of the measurement target (see Japanese Patent Application Laid-Open Nos. 2001-91247 and 2001-147110 and Jordi Pages, et. al., “Overview of coded light projection techniques for automatic 3D profiling” IEEE International Conference on Robotics and Automation, ICRA2003, pp. 133-138, Taipei, Taiwan, 14-19th September 2003).
Specifically, Japanese Patent Application Laid-Open No. 2001-91247 discloses a technique of using a random pattern in which brightness blocks having predetermined brightness gray levels are randomly arranged (namely, there is no regularity in the brightness change).
Japanese Patent Application Laid-Open No. 2001-147110 discloses a technique of using a non-periodic projecting pattern which has no periodicity in a dot size, line length, a line thickness, a position, density, and the like based on random numbers or normal random numbers. Japanese Patent Application Laid-Open No. 2001-147110 also discloses a technique of determining whether the projecting pattern is appropriate by evaluating the generated projecting pattern with a pattern change rate, a feature quantity, s dispersion value, pattern matching, and the like.
Page, et al. discloses various projecting patterns such as a projecting pattern based on an M-sequence pseudo-random sequence, a binary code pattern including black and white, and a projecting pattern expressed by gray scale or color.
In FIG. 12, a plane formed by a measuring point MP, an imaging point IP1 of the camera 1, and an imaging point IP2 of the camera 2 is called epipolar plane, and a straight line connecting the measuring point MP and each of the imaging points IP1 and IP2 is called epipolar line. When the positional relationship between the camera 1 and the camera 2 is already known, a direction of the epipolar line connecting the measuring point MP and the imaging point IP1 is determined on the image G2. Therefore, as is well known, the stereo correspondence problem can be performed only on the epipolar line.
However, in the conventional techniques using projecting patterns, for example, assuming that the window is formed by the pixel region having one-by-three pixels, sometimes a vector corresponding to one window becomes (50,43,23) in the taken image while a vector corresponding to the other window becomes (100,86,46).
Even if the linear shift is generated between the brightness values in the taken images, the linear shift can be absorbed using the normalized correlation function C (formula (1)). At the same time, in the normalized correlation function C, there is a risk that the stereo correspondence problem is faultily performed between the brightness values in the taken images, when a scalar multiplication relationship is generated between the brightness values in the taken images using the conventional projecting pattern.
This corresponds not only to the case where the normalized correlation function C (formula (1)) is used but to the case where an increment sign correlation is performed (see T. Kaneko, et. al., “Stereoscopic measurement of borehole wall three-dimensional shape by increment sign correlation” Journal of the Japan Society for Precision Engineering, vol. 67, no. 1, pp. 81-85, 2001).
Japanese Patent Application Laid-Open No. 2001-147110, proposes that after the pattern is projected to take the image, the taken image is evaluated. When the taken image does not correspond to a reference, another pattern is produced, and the projection, taking the image, and the evaluation are performed again. Therefore, the pattern appropriate pattern is not always generated from the beginning.